Credit Card segmentation with Machine Learning¶
Import Library¶
In [1]:
#Basic python library which need to import
import pandas as pd
import numpy as np
#Date stuff
from datetime import datetime
from datetime import timedelta
#Library for Nice graphing
import seaborn as sns
import matplotlib.pyplot as plt
import statsmodels.formula.api as sn
%matplotlib inline
#Library for statistics operation
import scipy.stats as stats
# Date Time library
from datetime import datetime
#Machine learning Library
import statsmodels.api as sm
from sklearn import metrics
from sklearn.cross_validation import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.ensemble import RandomForestRegressor
from sklearn.tree import DecisionTreeRegressor
from sklearn.ensemble import AdaBoostRegressor
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.svm import SVC, LinearSVC
from sklearn.metrics import mean_squared_error as mse
from sklearn.metrics import mean_absolute_error, mean_squared_error
# Ignore warnings
import warnings
warnings.filterwarnings('ignore')
# Settings
pd.set_option('display.max_columns', None)
np.set_printoptions(threshold=np.nan)
np.set_printoptions(precision=3)
sns.set(style="darkgrid")
plt.rcParams['axes.labelsize'] = 14
plt.rcParams['xtick.labelsize'] = 12
plt.rcParams['ytick.labelsize'] = 12
Load data¶
In [2]:
# reading data into dataframe
credit= pd.read_csv("CC_GENERAL.csv")
Information about data set¶
In [3]:
credit.head()
Out[3]:
In [4]:
credit.info()
In [5]:
# Find the total number of missing values in the dataframe
print ("\nMissing values : ", credit.isnull().sum().values.sum())
# printing total numbers of Unique value in the dataframe.
print ("\nUnique values : \n",credit.nunique())
In [6]:
credit.shape
Out[6]:
In [7]:
# Intital descriptive analysis of data.
credit.describe()
Out[7]:
a) Missing Value Treatment¶
- Since there are missing values in the data so we are imputing them with median.
In [8]:
credit.isnull().any()
Out[8]:
In [9]:
# CREDIT_LIMIT and MINIMUM_PAYMENTS has missing values so we need to remove with median.
credit['CREDIT_LIMIT'].fillna(credit['CREDIT_LIMIT'].median(),inplace=True)
credit['CREDIT_LIMIT'].count()
credit['MINIMUM_PAYMENTS'].median()
credit['MINIMUM_PAYMENTS'].fillna(credit['MINIMUM_PAYMENTS'].median(),inplace=True)
In [13]:
# Now again check the missing values.
credit.isnull().any()
Out[13]:
Deriving New KPI¶
1. Monthly average purchase and cash advance amount
Monthly_avg_purchase¶
In [10]:
credit['Monthly_avg_purchase']=credit['PURCHASES']/credit['TENURE']
In [12]:
print(credit['Monthly_avg_purchase'].head(),'\n ',
credit['TENURE'].head(),'\n', credit['PURCHASES'].head())
Monthly_cash_advance Amount¶
In [13]:
credit['Monthly_cash_advance']=credit['CASH_ADVANCE']/credit['TENURE']
In [14]:
credit[credit['ONEOFF_PURCHASES']==0]['ONEOFF_PURCHASES'].count()
Out[14]:
2- Purchases by type (one-off, installments)¶
- To find what type of purchases customers are making on credit card
In [45]:
credit.loc[:,['ONEOFF_PURCHASES','INSTALLMENTS_PURCHASES']]
Out[45]:
Find customers ONEOFF_PURCHASES and INSTALLMENTS_PURCHASES details¶
In [16]:
credit[(credit['ONEOFF_PURCHASES']==0) & (credit['INSTALLMENTS_PURCHASES']==0)].shape
Out[16]:
In [17]:
credit[(credit['ONEOFF_PURCHASES']>0) & (credit['INSTALLMENTS_PURCHASES']>0)].shape
Out[17]:
In [19]:
credit[(credit['ONEOFF_PURCHASES']>0) & (credit['INSTALLMENTS_PURCHASES']==0)].shape
Out[19]:
In [20]:
credit[(credit['ONEOFF_PURCHASES']==0) & (credit['INSTALLMENTS_PURCHASES']>0)].shape
Out[20]:
As per above detail we found out that there are 4 types of purchase behaviour in the data set. So we need to derive a categorical variable based on their behaviour
In [21]:
def purchase(credit):
if (credit['ONEOFF_PURCHASES']==0) & (credit['INSTALLMENTS_PURCHASES']==0):
return 'none'
if (credit['ONEOFF_PURCHASES']>0) & (credit['INSTALLMENTS_PURCHASES']>0):
return 'both_oneoff_installment'
if (credit['ONEOFF_PURCHASES']>0) & (credit['INSTALLMENTS_PURCHASES']==0):
return 'one_off'
if (credit['ONEOFF_PURCHASES']==0) & (credit['INSTALLMENTS_PURCHASES']>0):
return 'istallment'
In [22]:
credit['purchase_type']=credit.apply(purchase,axis=1)
In [23]:
credit['purchase_type'].value_counts()
Out[23]:
4. Limit_usage (balance to credit limit ratio ) credit card utilization¶
- Lower value implies cutomers are maintaing thier balance properly. Lower value means good credit score
In [52]:
credit['limit_usage']=credit.apply(lambda x: x['BALANCE']/x['CREDIT_LIMIT'], axis=1)
In [53]:
credit['limit_usage'].head()
Out[53]:
5- Payments to minimum payments ratio etc.¶
In [54]:
credit['PAYMENTS'].isnull().any()
credit['MINIMUM_PAYMENTS'].isnull().value_counts()
Out[54]:
In [55]:
credit['MINIMUM_PAYMENTS'].describe()
Out[55]:
In [56]:
credit['payment_minpay']=credit.apply(lambda x:x['PAYMENTS']/x['MINIMUM_PAYMENTS'],axis=1)
In [57]:
credit['payment_minpay']
Out[57]:
Extreme value Treatment¶
- Since there are variables having extreme values so I am doing log-transformation on the dataset to remove outlier effect
In [58]:
# log tranformation
cr_log=credit.drop(['CUST_ID','purchase_type'],axis=1).applymap(lambda x: np.log(x+1))
In [59]:
cr_log.describe()
Out[59]:
In [60]:
col=['BALANCE','PURCHASES','CASH_ADVANCE','TENURE','PAYMENTS','MINIMUM_PAYMENTS','PRC_FULL_PAYMENT','CREDIT_LIMIT']
cr_pre=cr_log[[x for x in cr_log.columns if x not in col ]]
In [61]:
cr_pre.columns
Out[61]:
In [62]:
cr_log.columns
Out[62]:
Insights from KPIs¶
Average payment_minpayment ratio for each purchse type.¶
In [63]:
x=credit.groupby('purchase_type').apply(lambda x: np.mean(x['payment_minpay']))
type(x)
x.values
Out[63]:
In [71]:
ax.barh?
In [72]:
fig,ax=plt.subplots()
ax.barh(y=range(len(x)), width=x.values,align='center')
ax.set(yticks= np.arange(len(x)),yticklabels = x.index);
plt.title('Mean payment_minpayment ratio for each purchse type')
Out[72]:
In [73]:
credit.describe()
Out[73]:
customers with installment purchases are paying dues¶
In [77]:
credit[credit['purchase_type']=='n']
Out[77]:
In [78]:
credit.groupby('purchase_type').apply(lambda x: np.mean(x['Monthly_cash_advance'])).plot.barh()
plt.title('Average cash advance taken by customers of different Purchase type : Both, None,Installment,One_Off')
Out[78]:
Customers who don't do either one-off or installment purchases take more cash on advance¶
In [79]:
credit.groupby('purchase_type').apply(lambda x: np.mean(x['limit_usage'])).plot.barh()
Out[79]:
Original dataset with categorical column converted to number type.¶
In [139]:
cre_original=pd.concat([credit,pd.get_dummies(credit['purchase_type'])],axis=1)
Preparing Machine learning algorithm¶
We do have some categorical data which need to convert with the help of dummy creation
In [80]:
# creating Dummies for categorical variable
cr_pre['purchase_type']=credit.loc[:,'purchase_type']
pd.get_dummies(cr_pre['purchase_type'])
Out[80]:
Now merge the created dummy with the original data frame¶
In [81]:
cr_dummy=pd.concat([cr_pre,pd.get_dummies(cr_pre['purchase_type'])],axis=1)
In [82]:
l=['purchase_type']
In [83]:
cr_dummy=cr_dummy.drop(l,axis=1)
cr_dummy.isnull().any()
Out[83]:
In [84]:
cr_dummy.info()
In [85]:
cr_dummy.head(3)
Out[85]:
In [86]:
sns.heatmap(cr_dummy.corr())
Out[86]:
- Heat map shows that many features are co-related so applying dimensionality reduction will help negating multi-colinearity in data
- Before applying PCA we will standardize data to avoid effect of scale on our result. Centering and Scaling will make all features with equal weight.
Standardrizing data¶
- To put data on the same scale
In [88]:
from sklearn.preprocessing import StandardScaler
In [89]:
sc=StandardScaler()
In [90]:
cr_dummy.shape
Out[90]:
In [91]:
cr_scaled=sc.fit_transform(cr_dummy)
In [ ]:
cr_scaled
Applying PCA¶
With the help of principal component analysis we will reduce features
In [94]:
from sklearn.decomposition import PCA
In [95]:
cr_dummy.shape
Out[95]:
In [96]:
#We have 17 features so our n_component will be 17.
pc=PCA(n_components=17)
cr_pca=pc.fit(cr_scaled)
In [97]:
#Lets check if we will take 17 component then how much varience it explain. Ideally it should be 1 i.e 100%
sum(cr_pca.explained_variance_ratio_)
Out[97]:
In [98]:
var_ratio={}
for n in range(2,18):
pc=PCA(n_components=n)
cr_pca=pc.fit(cr_scaled)
var_ratio[n]=sum(cr_pca.explained_variance_ratio_)
In [99]:
var_ratio
Out[99]:
Since 6 components are explaining about 90% variance so we select 5 components
In [100]:
pc=PCA(n_components=6)
In [101]:
p=pc.fit(cr_scaled)
In [102]:
cr_scaled.shape
Out[102]:
In [103]:
p.explained_variance_
Out[103]:
In [104]:
np.sum(p.explained_variance_)
Out[104]:
In [105]:
np.sum(p.explained_variance_)
Out[105]:
In [106]:
var_ratio
Out[106]:
In [107]:
pd.Series(var_ratio).plot()
Out[107]:
Since 5 components are explaining about 87% variance so we select 5 components
In [108]:
cr_scaled.shape
Out[108]:
In [109]:
pc_final=PCA(n_components=6).fit(cr_scaled)
reduced_cr=pc_final.fit_transform(cr_scaled)
In [110]:
dd=pd.DataFrame(reduced_cr)
In [111]:
dd.head()
Out[111]:
So initially we had 17 variables now its 5 so our variable go reduced
In [112]:
dd.shape
Out[112]:
In [113]:
col_list=cr_dummy.columns
In [114]:
col_list
Out[114]:
In [115]:
pd.DataFrame(pc_final.components_.T, columns=['PC_' +str(i) for i in range(6)],index=col_list)
Out[115]:
So above data gave us eigen vector for each component we had all eigen vector value very small we can remove those variable bur in our case its not.¶
In [117]:
# Factor Analysis : variance explained by each component-
pd.Series(pc_final.explained_variance_ratio_,index=['PC_'+ str(i) for i in range(6)])
Out[117]:
Clustering¶
Based on the intuition on type of purchases made by customers and their distinctive behavior exhibited based on the purchase_type (as visualized above in Insights from KPI) , I am starting with 4 clusters.
In [119]:
from sklearn.cluster import KMeans
In [120]:
km_4=KMeans(n_clusters=4,random_state=123)
In [121]:
km_4.fit(reduced_cr)
Out[121]:
In [ ]:
km_4.labels_
In [123]:
pd.Series(km_4.labels_).value_counts()
Out[123]:
Here we donot have known k value so we will find the K. To do that we need to take a cluster range between 1 and 21.
Identify cluster Error.¶
In [124]:
cluster_range = range( 1, 21 )
cluster_errors = []
for num_clusters in cluster_range:
clusters = KMeans( num_clusters )
clusters.fit( reduced_cr )
cluster_errors.append( clusters.inertia_ )# clusters.inertia_ is basically cluster error here.
In [125]:
clusters_df = pd.DataFrame( { "num_clusters":cluster_range, "cluster_errors": cluster_errors } )
clusters_df[0:21]
Out[125]:
In [126]:
# allow plots to appear in the notebook
%matplotlib inline
import matplotlib.pyplot as plt
plt.figure(figsize=(12,6))
plt.plot( clusters_df.num_clusters, clusters_df.cluster_errors, marker = "o" )
Out[126]:
From above graph we will find elbow range. here it is 4,5,6
Silhouette Coefficient¶
In [127]:
from sklearn import metrics
In [128]:
# calculate SC for K=3 through K=12
k_range = range(2, 21)
scores = []
for k in k_range:
km = KMeans(n_clusters=k, random_state=1)
km.fit(reduced_cr)
scores.append(metrics.silhouette_score(reduced_cr, km.labels_))
In [129]:
scores
Out[129]:
In [130]:
# plot the results
plt.plot(k_range, scores)
plt.xlabel('Number of clusters')
plt.ylabel('Silhouette Coefficient')
plt.grid(True)
In [131]:
color_map={0:'r',1:'b',2:'g',3:'y'}
label_color=[color_map[l] for l in km_4.labels_]
plt.figure(figsize=(7,7))
plt.scatter(reduced_cr[:,0],reduced_cr[:,1],c=label_color,cmap='Spectral',alpha=0.1)
Out[131]:
It is very difficult to draw iddividual plot for cluster, so we will use pair plot which will provide us all graph in one shot. To do that we need to take following steps
In [132]:
df_pair_plot=pd.DataFrame(reduced_cr,columns=['PC_' +str(i) for i in range(6)])
In [133]:
df_pair_plot['Cluster']=km_4.labels_ #Add cluster column in the data frame
In [134]:
df_pair_plot.head()
Out[134]:
In [135]:
#pairwise relationship of components on the data
sns.pairplot(df_pair_plot,hue='Cluster', palette= 'Dark2', diag_kind='kde',size=1.85)
Out[135]:
It shows that first two components are able to indentify clusters
Now we have done here with priciple component now we need to come bring our original data frame and we will merge the cluster with them.¶
To interprate result we need to use our data frame
In [136]:
# Key performace variable selection . here i am taking varibales which we will use in derving new KPI.
#We can take all 17 variables but it will be difficult to interprate.So are are selecting less no of variables.
col_kpi=['PURCHASES_TRX','Monthly_avg_purchase','Monthly_cash_advance','limit_usage','CASH_ADVANCE_TRX',
'payment_minpay','both_oneoff_installment','istallment','one_off','none','CREDIT_LIMIT']
In [137]:
cr_pre.describe()
Out[137]:
In [140]:
# Conactenating labels found through Kmeans with data
cluster_df_4=pd.concat([cre_original[col_kpi],pd.Series(km_4.labels_,name='Cluster_4')],axis=1)
In [141]:
cluster_df_4.head()
Out[141]:
In [142]:
# Mean value gives a good indication of the distribution of data. So we are finding mean value for each variable for each cluster
cluster_4=cluster_df_4.groupby('Cluster_4')\
.apply(lambda x: x[col_kpi].mean()).T
cluster_4
Out[142]:
In [143]:
fig,ax=plt.subplots(figsize=(15,10))
index=np.arange(len(cluster_4.columns))
cash_advance=np.log(cluster_4.loc['Monthly_cash_advance',:].values)
credit_score=(cluster_4.loc['limit_usage',:].values)
purchase= np.log(cluster_4.loc['Monthly_avg_purchase',:].values)
payment=cluster_4.loc['payment_minpay',:].values
installment=cluster_4.loc['istallment',:].values
one_off=cluster_4.loc['one_off',:].values
bar_width=.10
b1=plt.bar(index,cash_advance,color='b',label='Monthly cash advance',width=bar_width)
b2=plt.bar(index+bar_width,credit_score,color='m',label='Credit_score',width=bar_width)
b3=plt.bar(index+2*bar_width,purchase,color='k',label='Avg purchase',width=bar_width)
b4=plt.bar(index+3*bar_width,payment,color='c',label='Payment-minpayment ratio',width=bar_width)
b5=plt.bar(index+4*bar_width,installment,color='r',label='installment',width=bar_width)
b6=plt.bar(index+5*bar_width,one_off,color='g',label='One_off purchase',width=bar_width)
plt.xlabel("Cluster")
plt.title("Insights")
plt.xticks(index + bar_width, ('Cl-0', 'Cl-1', 'Cl-2', 'Cl-3'))
plt.legend()
Out[143]:
Insights
Clusters are clearly distinguishing behavior within customers¶
- Cluster 2 is the group of customers who have highest Monthly_avg purchases and doing both installment as well as one_off purchases, have comparatively good credit score. This group is about 31% of the total customer base
- cluster 1 is taking maximum advance_cash and is paying comparatively less minimum payment and poor credit_score & doing no purchase transaction. This group is about 23% of the total customer base
- Cluster 0 customers are doing maximum One_Off transactions and least payment ratio. This group is about 21% of the total customer base
- Cluster 3 customers have maximum credit score and are paying dues and are doing maximum installment purchases. This group is about 25% of the total customer base
In [149]:
# Percentage of each cluster in the total customer base
s=cluster_df_4.groupby('Cluster_4').apply(lambda x: x['Cluster_4'].value_counts())
print (s),'\n'
per=pd.Series((s.values.astype('float')/ cluster_df_4.shape[0])*100,name='Percentage')
print ("Cluster -4 "),'\n'
print (pd.concat([pd.Series(s.values,name='Size'),per],axis=1))
Finding behaviour with 5 Clusters:¶
In [ ]:
km_5=KMeans(n_clusters=5,random_state=123)
km_5=km_5.fit(reduced_cr)
km_5.labels_
In [151]:
pd.Series(km_5.labels_).value_counts()
Out[151]:
In [152]:
plt.figure(figsize=(7,7))
plt.scatter(reduced_cr[:,0],reduced_cr[:,1],c=km_5.labels_,cmap='Spectral',alpha=0.5)
plt.xlabel('PC_0')
plt.ylabel('PC_1')
Out[152]:
In [153]:
cluster_df_5=pd.concat([cre_original[col_kpi],pd.Series(km_5.labels_,name='Cluster_5')],axis=1)
In [154]:
# Finding Mean of features for each cluster
cluster_df_5.groupby('Cluster_5')\
.apply(lambda x: x[col_kpi].mean()).T
Out[154]:
Conclusion With 5 clusters :¶
we have a group of customers (cluster 2) having highest avergae purchases but there is Cluster 4 also having highest cash advance & secong highest purchase behaviour but their type of purchases are same.
Cluster 0 and Cluster 4 are behaving similar in terms of Credit_limit and have cash transactions is on higher side
So we don't have quite distinguishable characteristics with 5 clusters,
In [157]:
s1=cluster_df_5.groupby('Cluster_5').apply(lambda x: x['Cluster_5'].value_counts())
print (s1)
In [158]:
# percentage of each cluster
print ("Cluster-5"),'\n'
per_5=pd.Series((s1.values.astype('float')/ cluster_df_5.shape[0])*100,name='Percentage')
print (pd.concat([pd.Series(s1.values,name='Size'),per_5],axis=1))
Finding behavior with 6 clusters¶
In [ ]:
km_6=KMeans(n_clusters=6).fit(reduced_cr)
km_6.labels_
In [160]:
color_map={0:'r',1:'b',2:'g',3:'c',4:'m',5:'k'}
label_color=[color_map[l] for l in km_6.labels_]
plt.figure(figsize=(7,7))
plt.scatter(reduced_cr[:,0],reduced_cr[:,1],c=label_color,cmap='Spectral',alpha=0.5)
Out[160]:
In [169]:
cluster_df_6 = pd.concat([cre_original[col_kpi],pd.Series(km_6.labels_,name='Cluster_6')],axis=1)
In [170]:
six_cluster=cluster_df_6.groupby('Cluster_6').apply(lambda x: x[col_kpi].mean()).T
six_cluster
Out[170]:
In [171]:
fig,ax=plt.subplots(figsize=(15,10))
index=np.arange(len(six_cluster.columns))
cash_advance=np.log(six_cluster.loc['Monthly_cash_advance',:].values)
credit_score=(six_cluster.loc['limit_usage',:].values)
purchase= np.log(six_cluster.loc['Monthly_avg_purchase',:].values)
payment=six_cluster.loc['payment_minpay',:].values
installment=six_cluster.loc['istallment',:].values
one_off=six_cluster.loc['one_off',:].values
bar_width=.10
b1=plt.bar(index,cash_advance,color='b',label='Monthly cash advance',width=bar_width)
b2=plt.bar(index+bar_width,credit_score,color='m',label='Credit_score',width=bar_width)
b3=plt.bar(index+2*bar_width,purchase,color='k',label='Avg purchase',width=bar_width)
b4=plt.bar(index+3*bar_width,payment,color='c',label='Payment-minpayment ratio',width=bar_width)
b5=plt.bar(index+4*bar_width,installment,color='r',label='installment',width=bar_width)
b6=plt.bar(index+5*bar_width,one_off,color='g',label='One_off purchase',width=bar_width)
plt.xlabel("Cluster")
plt.title("Insights")
plt.xticks(index + bar_width, ('Cl-0', 'Cl-1', 'Cl-2', 'Cl-3','Cl-4','Cl-5'))
plt.legend()
Out[171]:
In [172]:
cash_advance=np.log(six_cluster.loc['Monthly_cash_advance',:].values)
credit_score=list(six_cluster.loc['limit_usage',:].values)
cash_advance
Out[172]:
Conclusion with 6 clusters:¶
- Here also groups are overlapping.
- Cl-0 and Cl-2 behaving same
Checking performance metrics for Kmeans¶
- I am validating performance with 2 metrics Calinski harabaz and Silhouette score
In [174]:
from sklearn.metrics import calinski_harabaz_score,silhouette_score
In [175]:
score={}
score_c={}
for n in range(3,10):
km_score=KMeans(n_clusters=n)
km_score.fit(reduced_cr)
score_c[n]=calinski_harabaz_score(reduced_cr,km_score.labels_)
score[n]=silhouette_score(reduced_cr,km_score.labels_)
In [176]:
pd.Series(score).plot()
Out[176]:
In [177]:
pd.Series(score_c).plot()
Out[177]:
Performance metrics also suggest that K-means with 4 cluster is able to show distinguished characteristics of each cluster.
Insights with 4 Clusters
- Cluster 2 is the group of customers who have highest Monthly_avg purchases and doing both installment as well as one_off purchases, have comparatively good credit score. This group is about 31% of the total customer base
- cluster 1 is taking maximum advance_cash and is paying comparatively less minimum payment and poor credit_score & doing no purchase transaction. This group is about 23% of the total customer base
- Cluster 0 customers are doing maximum One_Off transactions and least payment ratio and credit_score on lower side This group is about 21% of the total customer base
- Cluster 3 customers have maximum credit score and are paying dues and are doing maximum installment purchases. This group is about 25% of the total customer base
Marketing Strategy Suggested:¶
a. Group 2¶
- They are potential target customers who are paying dues and doing purchases and maintaining comparatively good credit score ) -- we can increase credit limit or can lower down interest rate -- Can be given premium card /loyality cards to increase transactions
b. Group 1¶
- They have poor credit score and taking only cash on advance. We can target them by providing less interest rate on purchase transaction
c. Group 0¶
- This group is has minimum paying ratio and using card for just oneoff transactions (may be for utility bills only). This group seems to be risky group.
d. Group 3¶
- This group is performing best among all as cutomers are maintaining good credit score and paying dues on time. -- Giving rewards point will make them perform more purchases.
In [ ]: